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Abstract. We show that the vakio function of a stochastic control problem 
is the unique solution of the associated Hamilton-Jacobi-Bellman (HJB) equa- 
tion, completely avoiding the proof of the so-called dynamic programming prin- 
ciple (DPP). Using Stochastic Perron's method we construct a super-solution 
lying below the value function and a sub-solution dominating it. A compari- 
son argument easily closes the proof. The program has the precise meaning of 
verification for viscosity-solutions, obtaining the DPP as a conclusion. It also 
immediately follows that the weak and strong formulations of the stochastic 
control problem have the same value. Using this method we also capture the 
possible face-lifting phenomenon in a straightforward manner. 



1. Introduction 

Stochastic Perron's method was introduced in [5] for hnear problems, and adapted 
in [T] to free-boundary problems associated to Dynkin games. In the present paper, 
we carry out a similar line of ideas, but with significantly different technicalities, to 
the most important case of non-linear problems, namely that of Hamilton-Jacobi- 
Bellman equations (HJB) in stochastic control. The result presented here actually 
represents the original motivation to introduce the stochastic version of Perron's 
method. 

The goal of the paper is 

(1) to prove the general result stating that "the value function of a control 
problem is the unique viscosity of the associated HJB" , 

(2) without having to first go through the proof of the Dynamic Programming 
Principle (DPP) but actually obtaining it as a by-product, 

(3) in an as much elementary manner as possible. 

The motivation for such goal is described in detail in [2] and [1]. To summarize, 
the program described by (1) and (2) (and, implicitly (3)) amounts to a verification 
result for viscosity solutions of HJB's. Overall, we believe this to be a genuinely 
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novel approach to stochastic control, that provides a deeper understanding of the 
relation between controlled diffusions and (viscosity) solutions of HJB's. 

In addition to being a new method to treat a fundamental class of problems, 
we believe the program carried out here has two notable features, which basically 
amount to achieving our goals above: 

(1) This is a direct/verification approach to dynamic programming, in that it 
first finds/constructs a solution to the HJB, then shows that such solution 
is the value function. Technically, this can be viewed as a probabilistic 
counterpart to the classical approach. 

(2) We believe it, indeed, to be elementary compared to going through the 
probabilistic proof of the DPP (which is often incomplete) and then having 
to prove comparison of viscosity solutions anyway. In particular, there 
is no need for us to use "conditional controls" or canonical spaces. These 
arguments are still needed even in the recent proof of Bouchard and Touzi of 
a weaker version of the DPP [3] . While measurable selection arguments are 
circumvented there through the weaker formulation, the Markov arguments 
mentioned above are still present. Our method consists only in applying 
Ito's formula along the smooth test functions for viscosity solutions, plus an 
elementary stopping argument. In addition, arguments of the same type 
as we use here (maybe even more complicated) have to be used anyway 
when one uses the weak DPP to prove that the value function is a viscosity 
solution. 

We present here a fresh look at a classic problem, so some comments on the existing 
literature are needed. We mention briefly only those works that are closest related, 
at the risk of omitting relevant but further ideas. 

We first start with some important work in stochastic control, which, in the same 
spirit as our paper, avoids the DPP. At a formal level, one of our main results. 
Theorem 14.11 looks very much like the main result in the seminal work of Fleming- 
Vermes [9] and (see also Remark l5.3|) . El More precisely, while the authors in [9] 
and [5] show that the value function is the infimum of classical super-solutions, we 
show that, the value function is below the infimum of stochastic super-solutions, 
which is a viscosity sub-solution. While appearing stronger than our Theorem 14.11 
(considered by itself, without the other main result Theorem l3.1|) , the notable result 
in Fleming- Vermes has two features: 

(1) It contains a sophisticated approximation/separation argument used on top 
of re-stating the optimization problem as an infinite dimensional convex 
program, 

(2) It still uses the very definition of the value function, 

(3) By itself, is not enough to show the value function is a viscosity sub-solution. 
Even if one does not mind the complicated approximation arguments, our 
Theorem HH] is still needed on top of the very strong results in [9] and [8] to 
get such a conclusion. Even combining Fleming- Vermes with the Perron's 
method in Ishii |10| would not yield this: the infimum over viscosity super- 
solutions may go below the value function, unless we now a a-priori that the 
value function is a viscosity sub-solution, and we also have a comparison 
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result (needed even for the viscosity version of Perron in jlOj). A sub- 
approximation counterpart to the work of Fleming- Vermes could close the 
argument, but this would still have a very different flavor than our work, 
since it uses, once again, the representation of the value function. Actually, 
the recent papers [5l|6l[3 carry along these lines, for path-dependent HJB's. 

If one attempts to only use the Perron's method in Ishii [10] to construct viscos- 
ity solutions, the same obvious obstacle described in relation to Fleming- Vermes 
arises: without additional knowledge on the properties of value function, it docs 
not compare with the output of Perron's method. 

It should be also mentioned how our result compares to existing results about 
verification for viscosity solutions of HJB's, namely [13]. The result in [T3| starts 
from the fact that the value function is the unique viscosity solution, and, using 
this piece of information, synthesizes the optimal control (if one exists) in terms 
of the generalized derivatives of the viscosity solution/ value function. Our result 
plays a role before the synthesis described in [13] , and proves exactly that the value 
function is the unique viscosity solution, without resorting to the use of DPP. 

The rest of the paper is organized as follows: In Section [21 we present the 
basic setup of the stochastic control problem, introduce the related HJB and the 
terminal condition. Moreover, we state our assumptions on the Hamiltonian. In 
Section |3l we consider the strong formulation of the stochastic control problem 
and introduce the class of stochastic sub-solutions via which we construct a lower 
bound on the value function which is a viscosity super-solution. In Section [4l we 
introduce the weak formulation of the stochastic control problem and introduce the 
class of stochastic super-solutions using which we construct a viscosity sub-solution 
to the HJB equation. Finally, in Section [S] we verify that both value functions, 
in the weak and the strong formulation, equal the unique viscosity solution using 
comparison. 

2. Setup 

Let [/ be a closed subset of R*^ (the control space) and O an open subset of R'^ 
(the state space). Let b : [0,T] x O x U ^ R'^ and a : [0,T] x O x U ^ Md,d' be 
two measurable functions. We consider the controlled diffusion 

(2.1) dXt = bit,Xt,Ut)dt + cT{t,Xt,ut)dWt, XeO. 

We assume that the state lives in the open domain O C M^, to include the treatment 
of utility maximization models for positive wealth, which is popular in mathemat- 
ical finance. Given a measurable function 5 : O — > R, our goal is to maximize the 
expected payoff received at a fixed time-horizon T > using progressively measur- 
able processes u taking values in U. Informally, we want to study the optimization 
problem 

supE[g{X^)], Xo = xe O. 

u 

Remark 2.1. We choose only to maximize terminal payoffs, just to keep the nota- 
tion simpler. In the literature, this is known as the Mayer formulation of stochastic 
control problems. The Bolza problem, which contains a running payoff as well, can 
be treated in an identical manner, with some additional notation. 
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One associates the following Hamiltonian to this problem: 

< t < T, xeO. 



H{t, x,p, M) := sup 



b{x, u) ■ p + —Tr{a{x, u)a{x, u)^ M) 



We make the following assumption on the Hamiltonian: 

Assumption 2.1. Let us denote the domain of H by 

dom{H) := {{t,x,p,M) G [0,T] x O x M'' x Sd ■ H{t,x,p,M) < oo}. 

We will assume that H is continuous in the interior of dom(H). Moreover, we will 
assume that there exists a continuous function G : [0, T] x O x R"^ x Aid ^ ^ such 
that 

(1) H{t,x, p,M) < oo =^ G{t,x,p,M) >0, 

(2) G{t,x,p,M) >0 =^ H{t,x,p, M) <oo. 

Remark 2.2. We should remark that our assumption above on the Hamilton- 
ian H differs from that of |12j . which assumes that the domain of H is closed. 
This latter assumption is well-suited for analyzing super-replication problems with 
volatility uncertainty but excludes the utility maximization problems. Our assump- 
tion works out well also for utility maximization problems, where O ~ (0, oo) and 
G{t,x,p,M) = -M. 

Using the Stochastic Perron's Method, our goal is to show that, when a compar- 
ison principle is satisfied, the value function is, immediately, the unique viscosity 
solution of 

(2.2) mm{-vtit,x) - H{t,x,v^{t,x),Vxx{t,x)),G{t,x,Vx{t,x),Vxx{t,x))} = 0, 
for {t,x) G [0, T) x O, with the terminal condition 

(2.3) u\m[v[T,x) - g{x),G{T,x,Vx[T,x),Vxx{T,x))]=Q, on O, 
without having to prove the dynamic programming principle. 

Remark 2.3. One may question why we do not impose any kind of boundary 
conditions on dO . This is because, as we can see from the assumptions below, we 
choose O as a natural domain, so that the controlled state process X never makes 
it to the boundary. 



3. Stochastic sub-solutions 

In this section we will consider the so-called "strong formulation" of the stochas- 
tic control problem. 

The main goal of the paper is to outline how the Stochastic Perron's Method in 
[2] and [1] can be used for the more important problem of Hamilton- Jacobi-Bellman 
equations. Having such goal in mind, but wanting to keep the presentation sim- 
pler, we make quite restrictive assumptions, without losing the very interesting case 
when a boundary layer is present. However, the restrictive assumptions we make 
are actually present in the important examples we have in mind. Our analysis 
can be carried through under weaker assumptions, but, as it is customary in sto- 
chastic control, this would have to be done on a case by case basis, adapting the 
method to the specific optimization problem. This is particularly important as far 
as admissibility is concerned. 
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Let {n, T , P) be a probability space supporting an -valued Brownian motion. 
Given T let F := {Tt-, < t < T} denote the completion of the natural filtration 
of this Brownian motion. (Note that F satisfies the usual conditions.) 

Assumption 3.1 (State Equation). For any {t,u) e [0,r] x U and x,y £ R''- 
we have 

\b{t,0,u)\ + \cr{t,0,u)\ < C{1 + \u\), 
^^■"^^ \b{t,x,u) - b{t,y,u)\ + \a{t,x,u) - (j{t,y,u)\ < L{\u\)\x - y\, 

for some constant C and some non- decreasing function L : [0, oo) [0, oo). 

In what follows, we will work with controls and solutions defined on stochastic 
intervals. It is well know that, for deterministic intervals, one can choose integrands 
which arc progressively measurable, optional or predictable, as they are equal up 
to equivalent classes. We choose here to work predictable controls, which are both 
the most general (i.e. work even for jump-diffusions) and best suited to handle 
joint-measurability in (t,a;) that is required on stochastic intervals. 

Admissibility (i.e. bounds or integrability) is another very important issue, and 
we choose here a very small class of admissible process, namely bounded controls, 
but the bound is not fixed a-priori (unless the control space U is itself compact). 
This allows to capture the full behavior of the value function, i.e. face lifting 
phenomenon, but the optimal control may not be admissible, if such a control 
exists. This choice of admissible controls is the same as in Section 6 of Krylov [TT] , 
for the case of unbounded controls. 

Definition 3.1. Let 0<T<p<Tbe stopping times. By Ur^p we denote the 
collection of predictable processes u : (t, p] — s- U , by which we mean that the joint 
map 

(0,T] {t,Uj) Ut(oj) X l{r{u,)<t<p{u)} 

is predictable with respect to the filtration F and which are uniformly bounded, i.e. 
there exists a positive constant < B(u) < oo such that 

\\u\\ := sup \ut{oj)\ < B{u). 

T(ui)<t<p(ui) 

Remark 3.1. Assumvtion \3.1\ on the controlled SDE, together with the Definition 
\3.1\ ensures that there is always a unigue strong ( adapted to Tt ) solution to the 
controlled SDE up to an explosion time. The additional Assumption Iff.iJI below 
actually means that there is never an explosion (for bounded controls). This is 
always the case if O = , or in the case of utility maximization, if the control is 
the proportion of stocks held. 

Assumption 3.2 (Natural Domain). For any stopping times t < p and any 
initial condition ^ G J> satisfying P(^ G O) = f , if u £ Ur,p, the unigue strong 
solution of the SDE 

dXt^bit,Xt,ut)dt + a{t,Xt,ut)dWt, T<t<p, 



(3.2) 



does not explode, i.e. P(X"'^'^ & O, t <t < p) = 1. 

We denote Z^o,t by U. Then let us define the value function by 
V(t,x)= sup E[5(X^'*''=)], <t <T, x eO. 
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The goal of this section is to construct a super-solution of the Hamilton-Jacobi- 
Bellman equation p.2p with the terminal condition p.3p that is smaller than the 
value function V. In order to do that, we need some growth property to be imposed 
on the pay-ofF function g and the potential solutions of the PDE. In this direction, 
we make an additional assumption: 

Assumption 3.3 (Growth in x). There exists a gauge function -0 : O — !• [0, cxa] 
such that 

(1) for any t < p and any initial condition ^ G J>, P(^ G O) = 1, which 
satisfies E['(/'(^)] < oo, if the control u is admissible, i.e. u e l^T,p, then 



(2) \g{x)\ < C(l + ^-(2;)) for some C. 

The assumption above is tailor-made to deal simultaneously with quadratic 
problems {O = M.'^,ip{x) = and utility maximization (O = (0, oo), ip{x) = 

a;P, -oo < p < I, p ^ 0). 

Definition 3.2. The set of stochastic sub-solutions for the parabolic PDE (|2.2p . 
denoted by V~, is the set of functions v : [0,T] x O ^ M- which have the following 
properties: 

(i) They are continuous and satisfy the terminal condition v{T, ■) < g{-) to- 
gether with the growth condition 

(3.3) \v{t,x)\<C{v){l+i){x)),Q<t<T, xeO, for some C{v) < oo. 

(ii) There exists a bound L{v) < oo, depending on v, such that for each stopping 
time T and each ^ G Jv such that P(^ G O) = 1 and E[?/'(^)] < oo, there 
exists a control u G Ur^T defined on [r, T] adapted to ¥ , satisfying the bound 
||w|| < L{v) and such that for any ¥-stopping time p G [r, T] we have that 



Remark 3.2. We ask for the sub-martingale property to hold only in between the 
fixed stopping time r and any later p > t, which is actually less than the full 
sub-martingale property on the stochastic interval [t,T]. 

Assumption 3.4. We assume that ^ 0. 

Remark 3.3. As sumption ] 3. 4\ is satisfied, for example, when g is bounded from 
below. 

A crucial property of the set of stochastic solutions is the following stability 
result: 

Proposition 3.1. If and are two stochastic sub-solutions, then v = \J 
is also a stochastic sub-solution. 

Proof. We will only show that v satisfies item (ii) of the definition of stochastic sub 
solution. We can choose the uniform bound corresponding to v as 



E sup i^iXl^) < oo ; 



T<t<p 



(3.4) 




L{v) = L{v^)y L{v^). 
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Now, fix a stopping time t and a random variable ^ G J> with G O) = 1 
and E[-0(^)] < oo. Then, by the definition of the stochastic sub-solutions and w^, 
it follows that there are two controls ||ui|| < L{v^) and ||w2| < i(w^) satisfying 

i;^(t,0 <E[«^(p,X;'-^-'«)|J-,], *G{1,2}. 

Now define a control u (on the stochastic interval (r, T] ) by 

(3.5) U = l[yl(^T-.^)>v-^{T,^)}Ul + l{iii(T-,C)<i'2(r,C)}W2- 

Now, for each t < p < T, we have 

(1) on {v^{t,S,) > w^(r, ^)} G J> we have 

(2) on {ij1(t, ^) < v^{t,^)} G J> we have 

Applying the definition of sub-solutions for and (for controls ui and U2) we 
get 



a.s., 



since {^^(t, ^) > v^(t, ^)} G J>. Therefore, according to item (1) above we have 



(3.6) l[.uHr,i}>v^T,i)}V^ [t, < 

Similarly, we obtain 

l{t)i(r,5)<«2(T-,5)}W^(r, ^) <E 
and by item (2) above we have 

(3.7) l|^y^r,i)<v^T,o}^'^i^'0 < 



l{t,i(T,?)>i;2(r,C)}1'(^p'^'^) 



a.s. 



a.s., 



a.s. 



Putting p.6p and (jSTTj) together we conclude. 



□ 



Theorem 3.1. (The suprcmum of stochastic sub-solutions is a viscosity super- 
solution) Let Assumptions \2.l\ (l). \3.1\ I j'.gj |g. j1 and \3.J\ hold true. Assume also 
that g is a lower semi- continuous function and V < 00. Then the lower stochastic 
envelope 

:— sup V <.V < 00, 



is a viscosity super-solution of (j2.2p . Moreover, if we define 
(3.8) v-{T-,x):^ liminf v-{t',x'), xeO, 

(t'<T,x')^{T,x) 

then the function v^(T—, •)(> g{-)) is a viscosity super- solution of 

Remark 3.4. The function v~ may not have a limit from the left at t ^ T. We, 
therefore, modify this function as described in (|3.8p at t ^ T. If we consider 
the function v~ with the new terminal condition w^(T— ,•), it still is lower-semi 
continuous, as it is used in the proof of Theorem \5.1\ 
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Proof. 

Step 1. The fact that < V follows dhectly from the definition of the class of 
stochastic sub-solutions and by the definition of Ut.r and V. 

Step 2. The interior super- solution property. Let (p : [0,T] x O ^ R"^ be a C^-^- 
test function such that — ip attains a strict local minimum equal to zero at some 
interior point (to,a;o) € (0,7) x W^. We first prove that 

(3.9) - vt{to,xo) - H(to,xo,v^Xto,xo),v^,j;{to,xo)) > 0, 
by contradiction. To this end, assume that 

{-ift - sup L"ifi){to,xo) < 0. 

u 

But then there exists u E U such that 

{-ipt~ Lftp){to,xo) < 0. 

Since the coefficients of the SDE are continuous there exists a small enough ball 
B{to,xo,e) such that 

-ift - L'tip{t, x) < 0, {t, x) e B{to, xo, s), 

and 

(p{t, x) < v~ (t, x), (i, x) G B{to, xo,£) - {{to, Xo)}. 

To be precise, all along the paper, we use the norm ||(t, a;)|| = max{|i:|, so 

B{to,xo,£) = {(t,x)\max{\t - to\, \x - xo\} < e}. 

Since v~ — p is lower semi-continuous and B{tQ,X(),e) — B{tQ,XQ,e /2) is compact, 
there exists a (5 > satisfying 

- 5 >Lp on B{tQ, Xo, e) - i?(to, a^o, e/2). 

Using Lemma 4.1 in [2] together with Proposition l3.1l abovc. we obtain a (countable) 
increasing sequence of stochastic sub-solutions w„ v~ . Now, since Lp is continu- 
ous, as well as w„'s, we can use a Dini argument (identical to the one in Lemma 2.4 
of [T]) to conclude that for 5' G (0,(5) there exists a stochastic sub-solution w = u„ 
(for some large enough n) such that 

(3.10) v~5'>p onB{to,xo,e)- B{to,xo,e/2). 
Choosing rj G (0, S') small enough we have that the function 

ip^ := ip + 1] 

satisfies 

-p^^ - L^p^t, x) < 0, (t, x) G B{ta,xo, £), 

p>'^{t,x) < v{t,x), {t,x) G B(to,xo,e) - B{to,xo,e/2), 

and 

p'^{tQ,xo) = v~(to,xo) +77 > v~{to,xo). 

Now we define 

^ I u V on B{to,xo,s) , 
\ w outside B(to, a;o, e). 

Clearly, v'^ is continuous and v'^{to,xo) = p'^{to,xo) > v~{to,xo). And since e can 
be chosen so that T > to + e, satisfies the terminal condition. In addition, the 
growth condition in (i) Definition l3.2l holds for v^, since such growth condition holds 
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for the approximate supremum v (although we may not have, without additional 
assumptions, a similar growth condition on v^). 

We only need to show that v"^ satisfies (ii) in Definition to get a contradiction 
and complete the proof. Let < r < T be a fixed stopping time and ^ G J>, 
e O) = 1, such that E[-0(^)] < oo. We need to construct a control u e Ur.r 
that works for w'' in (ii) in Definition 13.21 Following the arguments in the proof of 
Proposition 13.11 such a control u can be constructed in a surprisingly simple way, 
which represents a significant technical improvement over the previous work [2] or 

m- 

Denote by ito G Ut.t the control corresponding to initial time r and initial 
condition ^ in (ii) in Definition 13.21 for the stochastic sub-solution v. Denote by A 
the event 

A = {(t,0 e B(to,xo,e/2) and ^"(t,C) > v{t,0}- 
Define the new control admissible control ui G Ut^t by 

Ml = U X 1a + Mo X Ia<=, 

and by ti the first time after r when the diffusion started at ^ and controlled by 
ui hits the boundary of B{to, x'o, e/2): 

n = inf{T < t < T|Xi"^'^'« G dB{to,xo,e/2)}. 

Now, denote by 

=X;'^^'"'« G 95(^0, xo, e/2), 

and by U2 G Uti,t the control in (ii) in Definition 13.21 corresponding to v for the 
starting time ti and initial condition ^i. Now, we can finally define 

U = UiX l{^<t<^^} + U2 X l{^i<t<T}- 

Note that the control u is bounded by L{v) V \u\, and, therefore, it is admissible. 
Consider any stopping time p such that t < p < T. On the event A, (p^{X) is a 
martingale up to p A ti (because of Ito's formula together with the fact that ip"^ is 
bounded on the interior ball), which reads 

^"(^,0 <E[U^''(pATi,X,Tn«)|J-,] a.s. 

Since 

U^"(p A T,,X;j^) = U^"(p A n, X,T;f ) < Ui;"(p A ri, X.Xf ), 

we actually obtain 

U«''(t,0 = < ]E[lAM''(pATi,x;A;;f)|J-,] a.s. 

Next, we use the fact that ui is the sub-optimal control for v, together with v = 
everywhere outside the open ball B{to, xq, e/2), to obtain: 

1a^v^t,0 = lAMr,0 < E[lAMP^r,,XX;^)\Tr]=E[lA^v^pAn,X;-;,^^^)\J^r]. 
Putting the above together, we obtain: 



(3.11) w''(t,C)<E 



Let us introduce yet another notation: S = {p < ti} G Jv^ . We know that, on the 
boundary dB{tQ,XQ,e/2), v = w''. Applying the definition of u, together with the 
fact that U2 is sub-optimal for v starting at ri with condition , we have 

Ib^v^{ti,^i) = lBci;(Ti,a) < E[lBcz;(p,x;^^^i.«i)l-^rJ < E[lseZ;" (p, X^^^'?) | J",,] . 
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If we rewrite the RHS in as 



E 



pj\ri 



and use the tower property, we get, indeed 



a.s. 



This completes the proof of (|3.9p . from which it foUows that 
H{to,xo,ipx(to,xo),ipa;x(to,xo)) < oo. 
Thanks to Assumption 1 2 . 11 - ( 1 ) we also have that 

(3.12) G{to,xo,'Pxito,xo),ipxxito,xo)) > 0, 

finishing the proof of interior super-solution property. 

Step 3. The terminal condition, Part I. We will show that w^(T, ■) ~ g{-). 
Assume that for some xq G Mf^ we have 

v'{T,xo) < g{xo). 

We will use this information to construct a contradiction. Since g{-) is lower-semi 
continuous then there exists an e > such that 

g{x) > v~ {T,xo) + e, if — a-o|<e. 

Due to the fact that v~ is lower-semi continuous, it is bounded from below on the 
compact set 

{B{T,xo,e) - B{T, xo, e/2)) n ([0, T] x O). 
For a small enough ?y > we have that 

V {T,xo) — —<—s+ inf V {t,x). 

4?/ {t,x)eiB{T,xo,e)-B{T,xo,e/2))n([a,T]xO} 

Since the above inequality is strict, following the proof of Step 2 in Theorem 13. 1[ 
we use again Lemma 4.1 in [2] together with Proposition 13.11 above, and a Dini 
argument to find a stochastic sub-solution v G V~ such that 

(3.13) v-{T,xo) - — <-e+ inf v{t,x). 

4?/ (t,2;)e(B(T,xo,e)-B(T,2:o,e/2))n([0,rlxC)) 

For fc > define 

I l2 

^''■^^*^(i, x) = v-{T, xo) - - k{T - t). 



V 

Choose k large enough, at least as large as k > e/Ai] but possibly much larger, such 
that 



-(^4-" -supij'(^"-''^' 

u 

Using p.l3p we obtain 



(to,a;o)<0 on B{T,xo,e). 



+ « on (B(r,xo,e)-B(T,a;o,£/2))n([0,T] X O). 

On the other hand, 

^''■^^^(r,a;o)<i'"(T,xo)<g(T,xo)-£, for \x~xo\<e. 
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Now, let 6 < £ and define 



^e.,v,k,5 _ 1 vV jip''''''' + 5) on B {T,xo,e), 
\ V outside B{T,xo,e). 

Now using the idea in Step 1 of tlie proof, we can show that v^^^^^^^ G but 
w^'''''^'''(r, xo) ~ v^{T,xo) + ^ > v~{T,xo), leading to a contradiction. 

The only reason we actually proved v^{T, •) = g(-) was to get some information 
about the left liminf v^{T—, •). More precisely, since v~ is lower semicontinous, we 
know that 

g{-)^v-{T,-)<v-iT-,-). 

In order to finish the proof of the Theorem, we only need to show that v^{T—, ■) 
is a viscosity super-solution of (j2.3p . which we will do in the next step. 
Step 4. The terminal condition, Part II. We show that the l.s.c. function w^(T— , •) 
is a viscosity super-solution of 

G{T,x,v-{T,x),v-^{T,x))>0, x e O. 

We follow closely the proof of Lemma 4.3.2 in P^]. More precisely, we rely on the 
fact that v~ satisfies the same equation in the interior, a fact we established in Step 
2, to get information about the limit as t — >• T. Let y G M'' and i^{x) be a test 
function satisfying 

(3.14) = «-(T-,2/)-V'(y) = mm{v~ iT-,x) - yjix)). 

By the very definition of w~(T— , •), there exists a sequence (s™, converging to 
(T, y) with Sm < T such that 

lim v~{sm,yTn) = v^iT-,y). 

Let us construct another test function that depends both on t and x variables: 

T-t 



i)m{t,x) = ipix) - \x - y\ + 



(T 



and choose (i,„, Xm) G [s,„, T] x B{y, e) as a minimum of w~ —ipm on [s„i, T] xB{y, e) 
where e is chosen small enough so that B{y,e) C O. 

What we would like to do next is to show that in fact t,„ < T for large enough 
m and that Xm — > y- The first fact follows from the observation that 

W_(s™,y„i) - 1p„i{Sm,y,n) < -777777- 7 < 0: 

for large enough m and that 

V~{T-,x) - 'lprniT,x) > V~{T-,x) - ^l:{x) > 0, 

where the second inequality follows from p.l4p . Let us focus on the convergence 
of Xm to y. The sequence converges (up to choosing a sub-sequence) to some 
z G B{y,l). By construction, Sm < tm- Using this and the choice of (tm,Xm) we 



12 



ERHAN BAYRAKTAR AND MIHAI SIRBU 



obtain the following string of inequalities: 

< iv-{T-,z) - i;{z)) - {v-{T^, y) - ^(y)) 



< liminf 

< liminf 

m— >oo 



< liminf 



{v-{t y-m)) 



(l' (trn 1 ) V^m (^m ; •^m ) ) {^rrit Vm^ '4^m (^m i ym)) 



3^m - y\'^ + 



(T-5™)2 



which proves that z = y. 

We know that {tm,Xm) is a minimizer of ir — tp^ over [sm,T] x B{y,e) by 
definition, and we also know that «,„ < f;,„ < T for large m. Since Xm y, we 
conclude that (for m large enough) we have (v~ — 'ipm){tm,Xm) < {v^ — ipm){t,x) 
for tm ^ t < T, \x ~ Xm\ < e/2. While this does not mean that {tm,Xm) is a local 
interior min for v~ ~ tpra (because we may have tm = Sm), it does mean that we 
have a local "parabolic interior minimum" . It is well known that, for example from 
[4], for parabolic equations, a "parabolic interior minimum" is enough to use ipm 
as a test function at {trmXm), and, therefore first conclude that 

)-H{t ))>o, 

so H {tm, Xm, Dxip,n{ta, xo), Dl'iprnitrn, Xm)) < oo and. Consequently, 

Gi^tm T Xm , -^x'^-^mi^^m : Xm) : D,j.^jyi{im: Xm)) ^ 0- 

Now the claim of this step follows from the continuity of G and the fact that 
Xm y, as the derivatives of ipm with respect to x converge to those of 

□ 



4. Stochastic super-solutions 

In this section we consider the weak formulation of the stochastic control prob- 
lem. 

Assumption 4.1. We assume that the coefficients b : [0,T] xM.'^ xU -^R"^ and 
cr : [0, T] X X [/ ^ Md,d' (K) are continuous. 

Definition 4.1. For each {s,x) we denote bylAs,x the set of weak admissible controls 
for the (j2.ip . by which we mean a 

1^^...^ j,s.x^ (J-,^^^).<i<T, P^'^ {Wr)s<t<T, {Xns<t<T, iut)s<t<T 

where 

(1) (17'*'^, J^'''^, (J-j'*'^)s<t<T, P'*'^) is an arbitrary stochastic basis satisfying the 
usual conditions, 

(2) W^'^ is a d' -dimensional Brownian motion with respect to the filtration 

{Tt'''')s<t<T, 

(3) u is a predictable and uniformly bounded U -valued process, 
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(4) X"'^ is a continuous and adapted process satisfying (j2.1|) with initial con- 
dition Xs X e O, and P(Xt''^ e O, s < t < T) = 1 together with 



sup i^{xn 

s<t<T 



for the gauge function ip in Section\^ 

Now, for some measurable function g : O — > R, we denote by 
Q3(s,x) :=supE''-[5(Xj'")], 

the value function of the weak control problem. 

Assumption 4.2. The pay-off function g is an upper semi-continuous function 
satisfying |.g(-)| < C{l + i>{-)). 

Remark 4.1. (1) Because of the growth assumption on weakly controlled so- 
lutions, W'^'[g{X^^)\ is well defined and finite, so 2J > — oo. 
(2) When both are well-defined it clearly holds that < 03. 

Our goal in this section is to construct an upper bound of 2J that is a viscosity 
sub-solution. 



Definition 4.2. The set of stochastic super-solutions for the parabolic PDE (|2.2p . 
denoted by V^, is the set of functions v : [0, T] x — >■ M which have the following 
properties: 

(1) They are continuous and satisfy the terminal condition v{T, ■) > g{-) to- 
gether with the growth condition 

\v(t,x}\ < C(w)(l + V(a;)),0 <t<T, xeO. 

(2) For each {s,x) G [0,r] x O, and each weak control 

l^^s,.^ (^,^'-),<,<j., p^'-, {wr)s<t<T, {xns<t<T, K).<t<r) , 



the process {u{t,X^' ))s<t<T is a super-martingale on [W'^ ,P'^-^) with re- 
spect to the filtration (J-'^'^)s<t<T ■ 

Assumption 4.3. V+ / 0. 

Remark 4.2. Assumption \4.3\ is satisfied, for example, when g is bounded from 
above. 

Theorem 4.1. (The infimum of stochastic super-solutions is a viscosity sub-solution) 
Let Assumvtions [K7\ -(2), \4-^ anrf \4-^ hold true. Then v~^ = inf^,gy+ w is a 
viscosity sub-solution of (j2.2p . Moreover, the US C function v'^{T,-) is a viscosity 
sub-solution of (|2.3p . 

Proof. 

Step 1. The fact that ij+ > follows directly from the definition of the class 
of stochastic sub-solutions and by the definition of U. 

Step 2. The interior sub-solution property. Let (p : [0, T] x O — > M** be a C^'^-test 
function such that — ip attains a strict local maximum equal to zero at some 
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interior point {to,xo) G (0,r) x M"*, where the viscosity super-solution property 
fails, i.e., 

min{-v3t(to, xo)~H{t, x, ipx{to, xo),ipxx{to, xo)),G{t, x, ipx{to,xo),ipxx{to, xq))} > 0. 

Then since G is continuous and H is continuous in the interior of its domain it 
follows that there exists a small enough ball B{to, xq, e) such that, for all {t,x) £ 
B{tQ,xo,e) we have: 

inm{~ ipt{t,x) ~ H{t,x,(px{t,x),(pxxit,x)),G{t,x,ipxit,x),(pxx{t,x))} > 0. 

Now the rest of the proof of this step is very similar to the the corresponding step 
in the proof of Theorem 2.1 in [2], but much simplified by following the stopping 
idea in the proof of Theorem 13.11 (step 2) above. We still need to prove that the 
minimum of two stochastic super-solutions is a stochastic super-solution, and then 
use a Dini argument. However, this proof is actually easier than Step 2 in the proof 
of Theorem 13. II above, so we omit the details. 
Step 3. The boundary condition. 

Let Xq E O and ?/; be a smooth function on O such that 

= {T, Xq) ~ iIj{xo) ~ max(?j^(r, x) — •4!{x)). 

Assume, in addition, without losing generality that the maximum is strict. Let us 
assume, by contradiction, that 

(4.1) G{T,xo,i>xixo),i'xxixo)) > and w+(T, xq) > 5(2^0)- 

Since G is continuous, and, in addition, G is finite and continuous in the open set 
G > 0, we conclude that, there exists small e,5o > and a finite constant G such 
that 

H{t, X, ipxit, x), ipxx) < C, T - t < So,\x - xo\ < e. 
In addition, we also have (for small enough e) 

ipix) > g{x) + e, \x-xo\<e. 
Now, the whole idea is based on constructing a local super-solution 

ij''{t,x)=i^{x) + k{T-t) 

for large k, by decoupling the bounds S and e in the estimate above, then pushing it 
slightly down. Namely, we will make 5 much smaller than e. Fix 5^ and e as above. 
Denote by 

h{5)= sup (v+(t,x) - V'(a;)), < 5 < (5o. 

Interpreting ip as a, continuous function of two variables {t, x), which actually does 
not depend on t and taking into account that is USC, there exist a point where 
the maximum above is attained, i.e. 

h{6) = v'^{ts,X5) - Tp{xs). 

By compactness, we can subtract a sub-sequence (we still denote it as (5 \ 0) such 
that 

{ts,xs) ^ {T,x*), |<|x*-xo|<£. 
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Since is USC, we conclude that 

limsup/i((5) = hmsup lv~^{ts,xs) — tp{xs) 

< v+{T,x*)~iP{x*)< sup fu+(T,x) - < 0, 

^<\x — xo\<e 

where the last inequality follows from the fact that we have a strict max at xq and 
the last supremum is actually attained. Therefore, we can choose d < So small 
enough such that h{S) < 0. Now, for this fixed 6, with the notation 

6' = ~h{d) > 

we have 

(4.3) v+{t,x) <iIj{x) ~ S\ T-t<S,^<\x~xo\<e. 

Denote by D the compact "rectangular donut" 

D = {{t,x)\T-t< S, \x~xo\ <e}- {{t,x)\T-t< S/2, \x - xo\ < e/2}. 

Since, bu USC, u+ is bounded on {6/2 < T ~ t < 5,\x — Xo\ < s/2} wc can choose 
k large enough such that 

v+ < - 5' on {5/2 <T-t<5,\x-xo\< e/2}. 

Together with (|4.3p . we obtain 

?;+ < V'' - S' on D. 

In addition 

H{t,x,i;^^{t,x),'4j^^{t,x)) = H{t,x,il^.^{t,x),^^^{t,x)) < C, T-t < 6,\x-xo\ < e, 
so 

-V'f (t, x) - H{t, X, ^lit, x),^Ut, x))>k-C>0, 

for k even larger, iiT — t<6,\x — xo\ < e. Following the proof of Step 2 in Theorem 
13.11 we use again Lemma 4.1 in [2] and the Dini argument to obtain a stochastic 
sub-solution v £ V"*" such that 



v<ij'' - S'/2 on D. 
Now let 77 < (5'/2 < e and define 




vA[ip''-7]j T -t < S,\x - xo\ < e 
V otherwise 



It follows, using the same stopping argument as in the proof of Theorem 13. 1[ that 
v'' e V+. But wc also have that v''{T,xo) = v+{T,xq) - ij < v+{T,xo), which 
contradicts the definition of the function f +. 

□ 
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5. Verification by comparison 

Assumption 5.1. There is a comparison principle between USC sub-solutions and 
LSC super- solutions within the class \w\ < C(l + ijj) for the PDE 

(5.1) n\iii[w{x) - g{x),G{T,x,Xx{x),w^x{x))]^Q, on O. 



Lemma 5.1. Let us suppose that Assumvtion \5. 1\ and assumptions in both Theo- 
rem \3.1\ and Theorem\4-.l\ hold. Then: 



(5.2) v-{T-,-)=v+{T,-)=g{-), 

where g is the unique continuous viscosity solution of (|5.ip . In addition, both the 
strong and the weak value functions have well defined limits at T , equal to the 
terminal condition g: 

lim Vit,x')= lim Q3(t, a;') = o(x), a; e C 

(t<T,x')^(T,x) (t<T,x')^(T,x) 

Proof. It follows from their definitions that < v^. Since is USC, then 

(5.3) w~(T— ,x)= liminf v~{t,x')< limsup v'^{t,x') < v'^{T,x). 

{t<T,x')^{T,x) (t<T,x')^{T,x) 

Moreover, w^(T— ,•) is a LSC viscosity super-solution of (|5.ip as a result of The- 
orem [Ol and v^{T, ■) is an USC viscosity sub-solution of the same PDE due to 
Theorem 14.11 In addition, under the assumptions that both V~ and V"*" are non- 
empty, we have the bounds 

\v~l\v+\<C{l + 4'), 

obtaining therefore similar growth conditions for w^(r, •) and w^(T— , •). Thanks to 
the comparison assumption, it follows that v~^(T, ■) = v~ [T—, ■) and the common 
value is the unique continuous viscosity solution of (|5.ip that we denote by g. 
In order to prove the second statement, we only need to note that 

u~ < < QJ < w+ 



and plug the equality v-{T~, ■) = v+{T, ) = g{-) in (fO]) . 

□ 

Proposition 5.1. (G upper envelope of g.) Under Assumvtion \5.1[ the function g 
is the smallest (continuous) function above g which is a viscosity super- solution of 

(5.4) G{T,x,w.,{x),w^^{x)) =Q, on O. 

Proof. We know that g > g and that g is a viscosity super-solution of (|5.4p . Con- 
sider now a w > g and w is a super-solution of ()5.4|) . Then, ui is a super-solution of 
()5.1|) . Since g is a solution of (|5.ip and we have a comparison result, then g < w. 

□ 

Remark 5.1. When the space of controls is compact, one may take G to be equal 
to a positive constant. In that case g = g. 

Definition 5.1. We say that a comparison principle for (j2.2p holds if, when- 
ever we have an upper semi- continuous viscosity sub-solution v, and a lower semi- 
continuous viscosity super-solution w satisfying growth conditions \v\, \w\ < C{l-\-ip) 
with v{T, •) < w{T, •) on O, then v < w. 
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Now we are ready to state the main result of this section, which foUows as a 
corollary of Theorems 13.11 and 14.11 and Lemma 15.11 

Theorem 5.1. Let us assume that a comparison principle for ()2.2p holds. More- 
over, we assume that Assumption I5.il and assumptions in both Theorem \S.1\ and 
Theorem \4-l\ hold. Then, there exists a unique continuous (up to T) viscosity so- 
lution V G C([0,T] X O) of the PDE with terminal condition v{T, •) = g{-), 
satisfying the growth condition. Before time t we have: 



v{t,x)=v- {t, x) = v+ {t, x) = V{t, x) = 5J(i, x) {t, x) e[0,T)x O. 

Proof. Since Assumption 15.11 holds, then v~{T—,-) ~ v'^{T,-) = g{x). We now 
define the (still LSC) function 



w{t, x) 



v'{t,x), 0<t<T,xeO 
g{x), t^T,xeO. 



By definition, w < . At the same time, the function w is a LSC viscosity super- 
solution and is a USC viscosity sub-solution of (|2.2|) . Since w'^(T-) = w(T, •), 
so v'^ < w we can use comparison to conclude that < w, so 

v+ ^we C([0,T] X O). 

Denoting by v = w = , the proof is complete. □ 

Remark 5.2. When the controls are unbounded, the value function may display 
a discontinuity at the terminal time T, as the left limit is g, whereas the value 
at T is equal to g. (If t ^ T , it follows from the above theorem that the value 
function is continuous.) The discontinuity was already observed by Krylov in |llj 
on page 252, but the question of what the correct boundary condition should be was 
left open. Pham gave an answer to this question (for problems of super replication 
under volatility uncertainty type) in [12j on page 69 and gave a proof of the sub- 
solution property of the terminal condition using the DPP (see pages 71-75 of the 
same book). One of our contributions is to show that this boundary condition holds 
without relying on the DPP. The proof of the boundary condition comes out as a 
simple conclusion from the Stochastic Perron's method. 

Remark 5.3. (Comparison to Fleming- Vermes) As we have mentioned in the in- 
troduction, using our notation, Fleming- Vermes [9] and [8] prove that 

V ~ X) ^ iui{classic super- solutions}, 

under some technical assumptions (in particular, there is no boundary layer). The 
proof uses a sophisticated approximation/separation argument, and the probabilistic 
representation of V , 0. Even if one starts from here, it is not possible to conclude 
that V = V is a viscosity solution of the HJB ( even without the correction accounting 
for the infinite Hamiltonian) , directly, since one cannot prove that "inf over classical 
super- solutions is a viscosity sub-solution" . One would still need our Theorem 
to accomplish this, or, more direct and elementary, in our view, use the program 
we described above 



Theorem \3.1\ -h Theorem \4-. l[ h Comparison — > V" = D = unique viscosity solution 
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A comparison result is actually needed (with a separate analytic proof) anyway 
to obtain uniqueness of the solution, and, even more importantly, for the stability 
of numerical schemes. 
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